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ABSTRACT 


Wavelet  theory  provides  an  attractive  approach  to  signal  and  image  compression.  This 
work  investigates  a  new  approach  for  wavelet  transform  coefficient  selection  for  efficient 
image  compression.  This  approach  is  based  on  the  stochastic  optimization  of  wavelet 
scale  thresholds.  Experimental  results  are  compared  with  results  from  previously 
published  wavelet  image  compression  strategies. 

1.  Introduction 

Wavelets,  a  family  of  basis  functions,  are  dilations  and  translations  of  a  single  initial 
function  exhibiting  a  constant  shape.  Wavelets  originated  from  the  analysis  of  seismic  data 
by  J.  Morlet.  Wavelets  basis  sets  can  be  orthogonal  or  non-orthogonal,  and  each  wavelet 
can  have  compact  or  infinite  support.  Orthogonal  wavelets,  such  as  die  Haar  and  four- 
coefficient  Daubechies  wavelets,  provide  the  capability  of  decomposing  images  into 
multiresolution  representations  via  the  wavelet  transform1.  Wavelets,  via  the  wavelet 
transform,  provide  an  efficient  approach  for  the  compression  of  images. 

Traditionally,  image  transform  coding  or  compression  methods,  such  as  the  discrete 
cosine  transform,  rely  on  high  energy  compaction  with  an  nxn  passband  coefficient 
selection  criterion.  The  wavelet  transform  generally  results  in  poor  energy  compaction,  so 
an  alternative  strategy  is  required  for  wavelet  coefficient  selection.  This  paper  investigates 
a  stochastic  optimization  approach  for  wavelet  coefficient  selection. 

1.1.  Wavelet  Transform  Theory 

Performing  the  Fourier  transform  decomposes  an  image  into  a  frequency  representation 
and  allows  an  analyst  to  determine  if  a  particular  frequency  is  present,  but  does  not  provide 
any  spatial  information.  The  wavelet  transform,  on  the  other  hand,  results  in  a  spatial- 
frequency  representation  that  has  good  localization  in  both  domains.  Wavelets  can  be 
orthogonal  or  overcomplete,  but  since  we  are  interested  in  compressing  the  image,  we  do 
not  want  to  transform  the  image  into  more  coefficients  than  necessary  to  completely 
characterize  the  image.  Therefore,  we  use  an  orthogonal  wavelet  (the  Haar  wavelet)  in  this 
research. 

An  orthogonal  wavelet  is  determined  by  specifying  a  lowpass  filter  that  satisfies 

H( 0)  =  1 

\H(cof+\H(oj  +  Jtf  =  l  (1) 


Accuion  7ST 


NTIS  CRAAI 
OTIC  TAB 
Unannounced 
Justification 


J 

a 


By _ _ 

Distribution/ 


Availability  Codes 


Dtet 


m 


Avail  and/or 
Special 


l 


Proceedings  of  the  Third  Annua]  Conference  on  Evolutionary  Programming 


where  H(co)  represents  the  Fourier  transform  of  the  filter  h.  The  corresponding  highpass 
filter  g  is  defined  as 


G(co)  =  e~’mH\<Q  +  n)  (2) 

where  //*(•)  is  the  complex  conjugate  of  H(  ).  For  the  classic  orthogonal  wavelet,  the 
Haar,  h(Q)  =  1  ,  h(l)  =  1,  and  h  is  zero  everywhere  else.  The  interpolation  order  of  the 
wavelet  increases  with  the  length  of  the  wavelet  Wavelet  selection  can  affect  the  quality  of 
the  reconstruction.  We  are  interested,  however,  in  the  effects  of  stochastically  optimizing 
the  compression  thresholds,  and  therefore  a  single  wavelet  basis  will  be  used  in  this 
investigation. 

The  wavelet  transform  filters  the  data  and  results  in  a  lowpass,  or  blurred,  version  of 
the  original,  and  a  highpass,  or  detailed,  version.  The  lowpass  image  is  then  filtered  to 
produce  a  blurred  and  detail  image  of  it  Because  the  wavelet  transform  we  are  using  is 
separable,  the  image  wavelet  transform  can  be  implemented  as  successive  one-dimensional 
transforms  of  the  row  and  columns.  The  resulting  image  has  a  horizontal  detailed  image,  a 
vertical  detailed  image,  a  diagonal  detailed  image,  and  a  blurred  image. 

Given  a  signal  / ,  the  decomposition  is 

=  (3) 

=  -*)/?.,(*)  (4) 

k 

where  is  the  blurred  version  of  /  at  scale  J  and  /®is  the  detail  signal  at  scale  J .  One 

can  reconstruct  /  from  its  projections,  defined  as 

fj(n)  =  2%h(n  -  2k) fl,  (*)  +  2^g(n  -  2k) (k)  (5) 

k  k 

where  h(k)  -  h(-k)  and  g(k)  =  g(-k). 

1.2.  Wavelet  Compression  Techniques 

Image  compression  after  performing  the  wavelet  transform  is  achieved  by  selecting  a 
set  of  coefficients  from  the  wavelet  representation  to  keep.  Those  coefficients  not  selected 
are  set  to  zero.  The  number  selected  is  based  on  the  maximum  mean-squared  error  (MSE) 
acceptable,  the  compression  ratio  needed,  or  a  combination  of  both.  The  coefficients 
selected  are  then  coded  using  an  entropy-based  encoder  that  stores  the  value  of  the 
coefficient  and  its  location  within  the  current  scale. 

Different  sets  of  retained  coefficients  can  result  in  drastically  different  image 
reconstructions.  The  traditional  method  for  selection  is  simply  a  passband  filter  of  some 
height  and  width  that  passes  a  rectangular  region  of  the  transformed  data  and  sets  the  rest  to 
zero.  The  wavelet  transform  does  not  compact  energy  into  a  particular  region,  making  the 
passband  approach  suboptimal.  DeVore  et  al2  show  that  this  is  indeed  the  case,  and 
present  an  algorithm  that  keeps  the  highest  N  coefficients  and  discards  the  rest.  Argast  et 
al3  demonstrate  that  coefficients  from  lower  resolution  scales  affect  a  larger  part  of  the 
image  than  coefficients  in  the  high  resolution  scales.  Argast  presents  an  algorithm  for 
selecting  coefficients  based  on  a  logarithmic  threshold,  where  the  threshold  of  each  scale  is 
twice  the  threshold  of  the  previous  lower  resolution  scale.  This  compression  strategy  is 
given  as: 
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for  J  -  S  to  1  { 

for  each  element  k 

in  scale  J 

if  CR%  =  GOAL  then  stop 

else 

o  if  |c,|<2'r 

SCl  C  £  — 

ck  otherwise 

}  end 

where  ck  is  a  transform  coefficient,  CR%  is  the  current  compression  percentage,  7  is  a 
constant  weight  factor,  and  J  and  5  are  respectively  the  current  and  total  wavelet  scales. 
7  is  set  initially  at  a  constant  (0.01  in  their  paper),  and  adjusted  so  that  the  compression 
ratio  can  be  met  While  using  this  approach  results  in  a  lower  mean-squared  error  in  the 
reconstructed  images  for  a  given  compression  than  DeVore's  algorithm,  it  was  still  not 
known  if  it  was  the  optimal  set  of  thresholds.  In  this  paper,  we  attempt  to  answer  this 
question  by  stochastically  optimizing  the  threshold  values. 

1.3.  Stochastic  Optimization 

Since  the  1950's,  random  (or  stochastic)  search  techniques  have  been  used  for  function 
optimization.  Random  search  strategies  are  competitive  with  traditional  search  strategies 
(such  as  gradient  search  techniques)  when  the  cost  or  objective  function  is  expensive  or 
difficult  to  compute,  or  when  the  function  to  be  minimized  has  many  suboptimal  solutions 
(local  minima).  Other  advantages,  enumerated  by  Kamopp4,  include  the  ease  of 
programming,  inexpensive  realization  of  possible  solutions,  as  well  as  flexibility  in  the 
expression  of  the  criterion  function. 

Stochastic  optimization  techniques  are  based  on  either  a  single  point  or  multiple  agent 
algorithms.  Single  point  algorithms  include  the  random  walk  and  the  method  of  Solis  and 
Wets5.  Solis  and  Wets5  provide  convergence  proofs  for  single  agent  random  search 
strategies.  A  convergence  proof  for  the  evolutionary  programming  algorithm,  the  multiple 
agent  search  strategy  developed  by  Fogel6,  is  given  by  Fogel7.  Under  very  weak 
assumptions,  both  Solis  and  Fogel  respectively  show  that  a  single  and  a  multiple  agent 
random  search  technique  will  probabilistically  converge  to  a  region  about  the  minima  of  the 
function. 

To  increase  convergence  efficiency,  variants  of  the  EP  algorithm  have  been 
developed8-9.  This  work  imbeds  the  method  of  Solis  and  Wets  into  the  traditional 
evolutionary  programming  paradigm.  The  parent  models  not  only  produce  a  single 
offspring  by  way  of  the  stochastic  process,  but  are  also  modified  by  the  Solis  and  Wets 
algorithm.  This  approach,  in  the  nomenclature  of  Bach  and  Schwefel10  is  as  follows: 


k  =  0; 

initialize:  P(0 )  =  {tf,(0),...,a|l(0)} 
where  a,  =  (xy,  V/  e  {l,...n}) 

evaluate:  P( 0):  <t>(P(0))  =  {<t>(a,(0)) . d>(«„(0))} 

do  { 

mutate:  a'(k)  =  m'{aj(k ))  V/  e  {1 . p] 

evaluate  P'(Jfc):  <b(PW)  =  {<&(<(*)) . <&(«;(*))} 
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modify  parents:  P"  (k)  =  [a"(k)  =  $'(<*,(*))  V/  €  {l,...,/i}j 
select:  P(k  + 1)  =  s[t]{P"(k)\jP'(k)) 
k  =  k  +  \ 

}  while  (l(P(k))  *  true) 

The  population  is  instantiated  with  /i  individuals  aj  V/e{l,.../i}.  The  mutation 
operator  m'(  )  is  applied  to  each  individual,  according  to  the  random  perturbation  scheme 

a'^  +  gj-mP  (6) 

(7) 


where  n  is  the  dimensionality  of  a ,  and  <!>(•)  is  the  criterion  function. 

2.  Evolving  Compression  Strategies 

The  optimal  wavelet  compression  strategy  is,  of  course,  a  function  of  two  (interacting) 
aspects,  die  compression  ratio  and  the  image  quality  of-  the  compressed  image.  The 
criterion  function  $(-)  should  measure  these  two  aspects  of  the  compressed  image. 
Measuring  the  compression  rado  of  the  compressed  image  is  straightforward.  Although 
image  quality  is  a  subjective  facet  of  an  image,  a  widely  used  objective  measure  of  image 
reconstructive  quality  is  the  mean-squared  error  between  the  reconstructed  and  original 
images.  Maximizing  the  compression  ratio  penalty,  denoted  as  Pc,  while  minimizing  the 
reconstructed  MSE  provides  die  foundation  for  the  criterion  function: 

*(")  =  “  ri*f  +  pc  (8) 

m  i=i  t=i 


To  derive  the  form  of  Pc,  experiments  were  conducted  using  threshold,  linear,  and 
exponential  functions  to  compute  the  compression  penalty.  In  our  investigation,  Pc  was 

chosen  as  an  exponential  function  of  the  compression  ratio  percentage  CR%  and  the 
percentage  goal  (Eq.  (9)).  The  penalty  function  biases  the  optimization  algorithm  to 
guarantee  the  achievement  of  the  desired  compression. 


lOOe^00" 

< 

0 


if  CR%  <  Goal 
if  CR%  £  Goal 


(9) 


The  algorithm  used  to  zero  coefficients  is  given  as 


1 for  J  =  1  to  S 

for  each  element  kin  J 

C*.y  = ' 

o 

if  T(J) 

tc‘.i 

otherwise 

where  T(J)  is  the  stochastically  determined  threshold  for  the  wavelet  transform's  J  th 
scale.  Each  vector  T  therefore  represents  a  wavelet  compression  strategy,  and  the 
determination  of  the  optimal  strategy  is  the  goal  of  the  stochastic  optimization  process. 
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3.  Results 


Images  where  selected  from  the  Georgia  Tech.  University  image  library  at  gatech.edu. 
Wavelet  compression  strategies  were  evolved  for  three  contrast-enhanced  512  x  512  gray 
scale  images,  each  with  eight  bits  per  pixel.  Contrast  stretching  was  performed  on  each 
image  before  analysis.  For  each  image,  the  compression  ratio  was  set  to  50:1 
( GOAL  =  .02).  Figure  1  displays  the  MSE  and  compression  value  of  the  best  strategies 
for  the  first  100  iterations  of  the  optimization  process. 

For  all  images,  the  number  of  parent  strategies  in  the  stochastic  optimization  process 
was  set  to  five.  Although  five  is  low  for  the  typical  stochastic  optimization  problem,  the 
number  proved  to  be  more  than  adequate  for  determining  the  image  compression  strategies. 
This  does  not  mean  that  the  solution  space  was  a  simple  well,  as  local  minima  were 
discovered  in  the  experimental  process.  By  adjusting  the  value  for  the  maximum  allowable 
standard  deviation  in  the  Solis  and  Wets  algorithm,  all  local  minima  encountered  were 
escaped  by  the  stochastic  optimization  process.  For  all  images  tested,  the  stochastic 
optimization  process  converged  to  a  solution  within  500  iterations. 

The  threshold  values  of  the  optimal  strategy  (the  optimal  T  vector)  after  500  iterations 
are  respectively  graphed  and  tabulated  in  Figure  2  and  Table  1.  It  is  important  to  note  that 
the  values  for  the  lower  scales  (scales  0  through  3)  are  below  all  coefficients  in  those 
scales,  and  therefore  very  few,  if  any,  coefficients  in  diese  scales  are  zeroed.  It  is 
interesting  that,  for  the  higher  scales,  each  scale  value  is  approximately  double  the  value  of 
the  previous  scale  for  all  images  tested. 


Iteration  Iteration 

(Image  l)  (Image  2) 


Figure  1.  Stochastic  optimizing  behavior  for  first  100  iterations  for  each  image. 
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Figure  2.  Evolved  threshold  strategies  for  images. 


Table  1.  Thresholds  for  optimal  evolved  strategy 


Image 

flKDH 

1 

2 

3 

4 

5 

7 

8 

9 

Image  1 

4.9358 

1.6176 

0.2637 

0.5912 

1.7359 

3.1925 

6.5342 

12.707 

25.924 

50.522 

5.1373 

3.0283 

0.3044 

0.4055 

0.4272 

1.1291 

4.1380 

16.519 

44.780 

mm 

1.6479 

1.4099 

0.5161 

2.1493 

4.4556 

9.7358 

20.457 

32.621 

A  comparison  of  the  stochastically  determined  compression  strategy  obtained  for 
normal  and  contrast  versions  of  a  single  image  is  shown  in  Table  2.  The  same  between - 
scale  relationship  of  the  threshold  values  found  in  the  enhanced  images  is  displayed  in  the 
non-enhanced  image. 


Table  2.  Comparisons  of  threshold  values  for  same  image 


■E3T3H 1 

0 

J 

2 

3 

4 

5 

6 

7 

8 

9 

Normal 

3.9590 

1.6142 

0.7182 

0.5840 

1.5741 

4.1666 

9.2286 

16.775 

36.951 

Enhanced 

4.9358 

1  F6176 

0.2637 

0.5912 

1.7359 

3.1925 

6.5342 

12.707 

25.924 

50.522 

For  each  image,  a  comparison  of  the  MSE  of  the  reconstructed  images  using  the 
evolved  compression  strategy  and  the  algorithms  of  DeVore  and  Argast  was  performed. 
The  results  are  shown  in  Table  3.  As  shown,  for  all  images  compressed  and  restored,  the 
stochastic  strategy  outperforms  the  deterministic  algorithms. 


Table  3.  MSE  Comparison  for  Compression  Strategies 


Compression  Algorithm 

Image  I 

Image  2 

Image  3 

DeVore  (algorithm  b) 

586.187 

59.997 

277.525 

Argast 

247.282 

33.582 

160.840 

Stochastic 

28.786 

150.938 
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5.  Conclusions 

These  results  demonstrate  that  a  stochastic  optimization  technique  can  produce  optimal 
threshold-based  strategies  for  wavelet-based  image  compression.  For  all  images  tested,  the 
resulting  strategies  produce  results  exceeding  present  deterministic  algorithms.  More 
importantly,  a  stochastic  approach  can  provide  insight  into  the  relationship  between 
compression  coding  and  the  information  content  of  the  wavelet  scales. 

Future  research  in  stochastic  optimization  of  wavelet  compression  algorithms  exists  in 
the  determination  of  the  general  relationship  between  the  energy  of  the  image  and  the 
threshold  values  required  for  optimal  compression,  and  the  statistical  properties  of  the 
threshold  values  and  their  interscale  relationship.  Another  independent  area  of  research  is 
the  stochastic  optimization  of  the  wavelet  basis  function  selected  for  the  compression  and 
reconstruction  of  an  image.  Our  future  work  will  address  both  these  areas  of  research. 
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